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CHAPTER  1 

INTRODUCTION 

In  the  last  two  decades ,  the  Department  of  Defense 
(hereinafter  referred  to  as  DOD)  has  experienced  a  tremen¬ 
dous  Increase  in  the  amount  of  spending  for  procurement  of 
military  hardware.  Indeed,  the  amounts  are  difficult  for 
the  "average"  individual  to  comprehend .  The  following 
table  shows  expenditures  for  durable  military  equipment  as 
reported  by  the  United  States  Bureau  of  the  Census  (29:357): 


Year 

Expenditures 

(In  billions  of  then -year  $) 

1975 

13.7 

1976 

14.3 

1977 

16.8 

1978 

17.8 

1979 

21.7 

This  equipment  included  such  items  as  aircraft,  ships,  mis¬ 
siles,  sacellites,  and  other  minor  and  major  weapon  systems. 
This  increasing  spending  trend  is  expected  to  continue  in 
light  of  recent  proposals  to  expand  and  modernize  the  mili¬ 
tary  forces  of  the  United  States. 

Unfortunately,  it  appears  that  not  all  of  that  money 
is  being  used  as  effectively  as  the  taxpayers  might  desire. 
According  to  the  chief  executive  officer  of  the  Northrop 
Corporation,  the  General  Accounting  Office  found  that  80 
percent  of  the  country’s  past  major  weapon  systems  procure- 
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ment  programs  incurred  overruns.  In  fact,  some  of  the  pro¬ 
grams'  costs  increased  more  than  200  percent  over  the 
initial  targets  (19:482). 

These  overruns  resulted  from  a  variety  of  causes 
including  inflation,  changes  in  defense  priorities  and  poli¬ 
cies,  technological  changes,  design  changes,  and  mismanage¬ 
ment.  Of  course,  some  cost  overruns,  such  as  those  caused 
by  changes  in  defense  priorities,  are  not  a  result  of  prob¬ 
lems  in  the  acquisition  process;  but  some  of  the  excess 
costs  may  be  attributable  to  the  way  in  which  weapon  sys¬ 
tems  are  procured.  Inefficiency  on  the  part  of  the  govern¬ 
ment  and  on  the  part  of  defense  contractors  may  be  sources 
of  some  cost  overruns.  These  overruns  can  be  minimized. 

To  do  so,  however,  will  require  some  fundamental  changes  in 
the  way  weapon  systems  are  acquired. 

Concern  over  unwieldy  costs  within  DOD  permeates 
even  the  highest  levels  of  government.  When  Mr.  Frank 
Carlucci  became  the  Deputy  Secretary  of  Defense,  he  found 
individuals  at  all  levels  of  the  DOD  who  were  bothered 

i 

"with  the  cost  of  acquisition,  particularly  the  overhead 
and  indirect  costs,  and  with  our  inability  to  estimate 
costs  realistically  [3:56]."  In  fact,  his  observations 
prompted  him  to  write: 

At  the  top  of  my  list  of  problems  in  the  DOD,  and 
the  Defense  Industry,  is  excessive  cost — both  from 
the  standpoint  of  what  we  get  from  the  dollars  we 
spend  and  what  we  spend  that  the  critics  think  is 
unnecessary.  Some  cost  increases  are  unavoidable, 
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and  we  must  face  up  to  factors  such  as  inflation,  ad¬ 
vances  in  the  state-of-the-art,  and  unavoidable  changes 
in  production  schedules.  However,  we  will  continue  to 
work  hard  to  cut  back  on  those  costs  which  can  be  con¬ 
trolled  [6:109]. 

Mr.  Carlucci's  views  are  reflected  in  his  32  initia¬ 
tives  (see  Appendix  A)  which  were;  in  a  large  part,  directed 
towards  reducing  acquisition  costs  (5:3).  Perhaps  concern 
about  military  spending  and  procurement  activities  is  best 
reflected  by  the  Honorable  William  Proxmire  who  stated  that 
"anyone  who  believes  we  are  getting  our  tax  dollars'  worth 
out  of  military  spending  is  the  real  victim  of  enchanting 
illusion  [30:2591]." 

Given  that  there  is  a  noticeable  amount  of  consterna 
tion  concerning  DOD  spending,  the  government  must  continue 
trying  to  improve  its  procurement  techniques  to  reduce 
spending  and  eliminate  inefficiency.  Traditional  techniques 
of  weapon  system  acquisition  use  historical  cost  figures  as 
a  basis  for  determining  what  new  weapons  systems  should 
cost.  After  taking  into  consideration  such  factors  as 
inflation,  historical  cost  trends,  and  design  changes,  the 
government  formulates  a  basis  for  negotiation.  Unfortu¬ 
nately,  this  method  of  doing  business  perpetuates  and  mul¬ 
tiplies  the  effects  of  wasteful  and  expensive  contractor 
practices  (13:5);  but  the  elimination  of  this  procurement 
approach  is  sure  to  meet  with  opposition  from  the  defense 
industry.  Dr.  J.  Ronald  Fox,  in  his  book  Arming  America: 

How  the  U.S.  Buys  Weapons,  stated: 
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Our  current  approach  to  pricing  results  in  reward¬ 
ing  contractors  for  inefficiency.  Typically  profits 
are  based  on  cost,  resulting  in  reverse  incentive  to 
cost  reduction.  When  the  government  is  unable  to 
determine  how  much  a  weapon  system  should  cost,  there 
is  little  pressure  on  the  producer  to  reach  the  high¬ 
est  level  of  efficiency.  His  costs  tend  toward  the 
government's  upper  budgeting  limit.  Past  cost  experi¬ 
ence — often  the  most  convenient  standard  for  measuring 
efficiency— becomes  a  misleading  indicator  of  future 
costs  [11:80]. 


In  order  to  obtain  more  hardware  for  the  dollar,  a 
number  of  innovations  have  surfaced.  Some  of  the  tech¬ 


niques  the  government  utilizes  to  achieve  cost  reductions 
include  Piecost,  Cost/Schedule  Control  Systems  Criteria, 
Multi-Year  Procurement,  Award  Fee,  Design-to-Cost,  and 
Integrated  Logistics  Support  (4:45-55).  Another  much  pub¬ 
licized  technique  is  known  ns  "Should  Cost".  There  is 
optimism  that  this  approach  can  generate  greater  returns 
for  each  tax  dollar  spent.  In  1973,  the  former  Comptroller 
General  of  the  United  States,  Elmer  B.  Staats,  stated, 
"there  is  great  potential  for  the  government  to  benefit 
from  the  proper  application  of  Should  Cost  concepts 
[30:2551]."  Additionally,  Lt.  Gen.  H.  H.  Driessnack  noted 
that  Should  Cost  provided  a  mechanism  to  keep  new  weapon 
system  costs  within  allowable  limits  (8) .  Should  Cost 
received  probably  its  strongest  indorsement  from  General 
Robert  T.  Marsh  in  January  of  1983.  At  that  time,  General 
Marsh,  Commander  of  Air  Force  Systems  Command,  stated  his 
intent  to  utilize  Should  Cost  as  part  of  his  "war  on  costs" 
to  control  areas  which  have  historically  shown,  or  are  now 
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thought  to  have,  a  high  probability  for  dramatic  changes 
during  the  life  of  the  cost  estimate  (1:8). 

Should  Cost 


Definition 


The  Should  Cost  idea  is  officially  defined  in  AFP 


70-5  as: 


A  technique  of  contract  pricing  that  employs  an 
integrated  team  of  government  acquisition/  contract/ 
administration/  audit,  and  engineering  representatives 
to  conduct  a  coordinated,  indepth  cost  analysis  at  the 
contractor's  or  subcontractor's  plants.  The  objective 
is  to  identify  uneconomical  or  inefficient  practices  in 
the  contractor's  management  and  operations  and  to  quan¬ 
tify  the  findings  in  terms  of  their  impact  on  cost. 

The  result  is  the  development  of  a  realistic  price 
objective  that  reflects  reasonably  achievable  economies 
and  efficiencies  [31:1-1]  . 

This  definition  is  almost  identical  to  that  found  in  the 
Defense  Acquisition  Regulation.  In  more  concise  terms, 
"Should  Cost  is  the  determination  of  the  amount  that  a 
product  ought  to  cost,  not  will  cost,  based  on  attainable 
efficiency  and  economy  of  operations  [14:13]  ." 


Concepts 


To  arrive  at  these  standards  of  efficiency  and 
economy,  a  Should  Cost  team  is  assembled  by  the  buying 
activity  to  investigate  the  various  functions  of  a  contrac¬ 
tor's  operations.  These  functions  are  usually  broken  down 
and  categorized  into  three  main  functional  areas:  manage- 
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ment  systems;  purchasing  and  pricing  systems;  and  engineer¬ 
ing,  technical,  and  manufacturing  systems.  By  having 
inefficient  contractor  practices  identified  for  them, 
government  contract  negotiators  may  be  in  a  position  to 
show  potential  contractors  where  cost  reductions  could  be 
realized.  The  DOD  can  then  be  more  confident  it  is  paying 
for  the  hardware  it  receives,  and  not  financing  wasteful 
contractor  operations.  For  example,  a  Should  Cost  analysis 
may  uncover  manufacturing  operations  in  which  excessive 
manual  labor  hours  are  applied  or  required.  By  eliminating 
this  particular  practice,  DOD  will  save  money  not  only 
through  fewer  direct  labor  hours  paid  for,  but  also  through 
a  lower  allocation  of  overhead,  which  is  usually  calculated 
on  a  per-direct-labor-hour  basis.  In  addition  to  the  bene¬ 
fits  to  the  DOD,  the  identification  and  correction  of  inef- 
fiencies  can  help  to  improve  the  competitive  position  of 
the  contractor.  A  more  efficient  operation  means,  basi¬ 
cally,  that  less  input  is  required  to  achieve  a  given  level 
of  output,  or  that  more  output  can  be  achieved  with  a  given 
level  of  input.  A  contractor  who  eliminates  inefficiencies 
identified  by  government  Should  Cost  analyses  can  reduce 
expenses,  and  thus  increase  his  bottom  line.  The  govern¬ 
ment,  by  performing  Should  Cost  analyses,  in  effect  assumes 
the  role  of  consultant.  The  astute  reader  will  probably 
recognize  that  accomplishing  a  thorough  Should  Cost  analy¬ 
sis  can  be  expensive.  Trained  personnel  must  spend  many 
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man-hours  in  the  contractor's  plant  reviewing  his  manage¬ 
ment  and  production  practices. 

Each  of  the  three  functional  areas  previously  men¬ 
tioned  is  usually  analyzed  by  a  separate,  specialized  team. 
Some  of  the  typical  activities  each  team  investigates  within 
the  functional  areas  are  as  follows  (36:35): 

Management  Systems  -  searches  out  areas  of  dupli¬ 
cation  and  overstaffing;  develops  interview  techniques 
for  all  team  members  to  assist  in  finding  the  areas  of 
greatest  potential  savings. 

Purchasing  and  Pricing  -  reviews  and  evaluates  usage 
factors,  contractor's  purchasing  system,  warehousing 
distribution,  inventory  control,  make -or -buy ,  vendor 
management,  scrap  control,  and  various  costing  factors; 
prepares  a  final  report  for  each  task  assigned;  com¬ 
putes  price  objectives. 

Engineering,  Technical,  and  Manufacturing  -  evalu¬ 
ates  direct  and  indirect  labor,  labor  standards,  wage 
and  salary  scales;  evaluates  fabrication,  assembly, 
testing,  and  inspection  standards;  evaluates  costing 
factors;  evaluates  variance  analysis  and  productivity 
standards . 

The  specific  activities  each  team  analyzes  will  be  dictated 
according  to  circumstances.  The  Should  Cost  concept  is 
not  standardized  to  the  point  where  all  teams  are  identical 
and  accomplishing  cookbook  procedures.  However,  the  follow¬ 
ing  skills  are  generally  needed  on  a  Should  Cost  team 


Industrial  Engineering 

Production  Engineering 

Design  Engineering 

Quality  Assurance 

Accounting 

Auditing 

Pricing 

Management  Analysis 
Mathematics  or  Statistics 
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(31:42) : 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 
(9) 
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Because  each  Should  Cost  team  is  tailored  for  the 
job  at  hand,  the  time,  cost,  and  personnel  requirements 
vary.  One  study  conducted  by  the  United  States  Army  stated 
that  the  team  size  is  usually  about  20  people  and  that  a 
thorough  Should  Cost  analysis  may  require  three  to  four 
months  from  initiation  to  completion  (2:43) .  In  a  paper 
done  for  the  Aeronautical  Systems  Division,  USAF  Major 
Gerald  Heuer  stated  that  "a  typical  [Should  Cost]  study 
would  pass  through  .  .  .  seven  phases  [17:213]."  He  went 
on  to  say  that  several  months  are  usually  required  to  com¬ 
plete  a  study,  of  which  six  to  eight  weeks  are  actually 
spent  at  the  contractor's  facility  (17:212-213).  Dr.  Herbert 
Hoehl,  Professor  of  Procurement  Management  at  the  Air  Force 
Institute  of  Technology,  reported  that  the  teams  may  consist 
of  anywhere  from  10  to  40  personnel  (18:49) .  These  figures 
serve  to  highlight  the  fact  that  Should  Cost  studies  involve 
significant  effort  and  cost.  In  general,  therefore.  Should 
Cost  should  be  applied  only  to  major  procurement  efforts 
where  the  potential  savings  may  outweigh  the  costs. 

In  addition  to  the  costs  of  the  studies,  there  is 
another  potential  drawback  to  application  of  Should  Cost. 

By  its  very  nature,  the  Should  Cost  method  requires  an  in¬ 
trusion  by  outsiders  into  a  contractor's  facility.  As  such, 
it  may  be  criticized  as  impinging  upon  the  contractor's 
private,  internal  affairs.  Accordingly,  as  one  Air  Force 
price  analyst  has  pointed  out,  the  DOD  may  be  told  to  either 
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accept  the  contractor's  proposal  or  take  its  business  else- 
where  (10:136);  but  Mr.  Gordon  Rule,  former  assistant  to 
the  head  of  the  Naval  Materiel  Command,  replied  to  those 
types  of  criticisms  in  the  following  manner: 

The  "Should  Cost"  Method  of  pricing  must  not  be 
construed  as  an  attempt  on  the  part  of  the  government 
to  tell  a  contractor  how  to  conduct  his  operation.  If, 
for  example,  a  contractor  wishes  to  conduct  a  patently 
inefficient  operation  with  excess  indirect  employees, 
poor  estimating,  labor  that  consistently  fails  to  meet 
standards,  lack  of  proper  competitive  subcontracting, 
abnormal  spoilage  and  rework,  etc.,  that  is  his  busi¬ 
ness.  It  is  the  government's  responsibility,  however, 
not  to  pay  taxpayer's  money  for  demonstrable  ineffi¬ 
ciencies  in  the  manufacturing  process  of  a  sole  source 
supplier,  regardless  of  the  quality  of  the  ultimate 
product  [10:144]. 

It  is  not  enough  that  the  government  select  the  most  advan¬ 
tageous  proposal.  Should  Cost  analysis  attempts  to  ensure 
that  as  little  waste  as  possible  is  incurred  in  system 
acquisitions. 


Assumptions 


The  preceding  review  of  the  Should  Cost  concept 
indicates  that  use  of  this  acquisition  technique  requires 
some  explicit  assumptions  be  made.  They  are  as  follows 
(2:42)  : 


-  Whether  due  to  complacency  on  the  part  of  con¬ 
tractors,  the  government,  or  both,  defense  contrac¬ 
tors  have  generally  become  inefficient  producers. 

-  Defense  contractors'  proposals  are  high  rela¬ 
tive  to  the  price  that  would  prevail  if  they  were 
efficient  producers. 
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-  Government  analysts  are  better  trained,  more 
knowledgeable,  more  objective  and/or  more  dedicated 
to  achieving  more  for  the  defense  dollar  than  are 
their  counterparts  in  the  defense  industry. 

-  Government  analysts  conducting  Should  Cost  studies 
will  identify  inefficiencies  and  use  the  findings  to 
support  government  contract  negotiating  positions. 

-  Should  Cost  studies  will  improve  the  weapons 
acquisition  process  by  achieving  lower  costs. 

Past  Studies  and  Results 

Overview 

The  first  Should  Cost  study  took  place  in  1967  at 
Pratt  &  Whitney  and  was  conducted  by  the  Navy.  It  was 
initiated  when  the  estimated  cost  of  each  F-lllB  engine, 
the  TF-30,  rose  from  an  original  estimate  of  $273,910  to 
$900,000  (10:138).  These  figures  represent  nominal  in¬ 
creases  over  the  five-year  period  from  1962  to  IS 67.  How¬ 
ever,  even  allowing  for  inflation,  which  was  relatively 
low  during  the  mid-1960s,  this  was  a  substantial  overrun. 
Since  that  time.  Should  Cost  has  received  a  great  deal  of 
publicity  praising  the  technique  as  the  panacea  for  the 
government's  out-of-control  acquisition  activities.  How¬ 
ever,  analyses  of  the  results  of  some  of  the  procurement 
efforts  involving  Should  Cost  have  yielded  mixed  results. 

In  this  section,  some  of  those  analyses  will  be  considered. 
Other  Should  Cost  topics  discussed  are  effects  of  timing 
on  studies,  criteria  for  studies,  and  size  of  the  teams. 


The  objective  of  Should  Cost,  as.  stated  by  the  U.S. 
Air  Force,  is  to  provide  the  DOD  with  empirically  derived, 
quantitative  data.  The  data  are  then  used  for  guidance  in 
contract  negotiations  (31:2-1).  This  objective  proved  to 
be  a  common  factor  among  the  literature  reviewed.  Because 
of  the  pragmatic  nature  of  the  data,  the  Should  Cost  con¬ 
cept  is  highly  oriented  toward  the  final  product  and  applied 
production  processes  of  the  vendor.  As  such,  the  amount  of 
cost  reduction  in  a  procurement  directly  attributable  to 
the  Should  Cost  study  is  dependent  upon  the  acquisition 
phase  in  which  the  study  is  completed.  Obviously,  as  more 
design  changes  are  incurred  by  a  product,  its  production 
processes  and  materials  will  also  change  and  the  validity 
of  the  study  will  decrease.  A  study  done  in  the  latter 
stages  of  a  system's  life  cycle  will  be  subject  to  less 
technological  and  design  uncertainty  than  a  study  conducted 
in  the  initial  stages.  Thus,  more  effective  measurement 
and  evaluation  of  the  input/output  relationships  can  be 
realized  (20:13).  Consequently,  the  later  studies  will 
better  demonstrate  the  possibilities  of  achieving  reduced 
costs.  Figure  I  is  a  graphical  representation  of  this  con¬ 
cept  (14:15).  The  exact  shape  of  the  curve  in  Figure  I 
is  unknown,  but  the  approximation  shown  here  indicates  that 
contract  negotiation  should  occur  as  late  as  possible  to 
reap  the  full  benefits  of  a  Should  Cost  study. 


Conceptual  Validation  Full-Scale  Initial  Follow-on 
Phase  Phase  Development  Production  Production 
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licability  of  Should  Cost  as  a  Function 
f  Technological  Certainty 


The  timing  of  the  study  is  not  the  only  factor  to 
consider  before  implementation.  The  expense  required  in 
conducting  a  Should  Cost  analysis  should  be  of  major  con¬ 
cern.  Various  sources  indicated  the  important  role  of  the 
following  criteria  in  determining  Should  Cost  applicability 
(4:47;  7:21?  31:2-2): 

-  Is  the  program  a  major  one  of  high  dollar  value? 

-  Does  the  contractor  have  substantial  amounts  of 
government  sales? 

-  Does  the  contractor  have  a  history  of  increasing 
costs? 

-  Will  there  be  a  significant  number  of  follow-on 
production  contracts? 

-  Is  there  adequate  time  for  the  study? 

Army  Materiel  Command  Pamphlet  715-7  states  that  a 
lack  of  adequate  price  competition  (as  in  the  case  of  a 
sole-source  procurement)  in  contracting  activities  should 
be  a  requirement  for  Should  Cost  studies  (34:2-1).  This  is 
based  on  the  view  that  when  multiple  suppliers  are  avail¬ 
able: 

There  will  be  some  form  of  price  competition  among 
suppliers.  Furthermore ,  if  price  competition  exists , 
efficiency  of  operations  and  reasonable  price  proposals 
are  assumed  to  result  [7:21]. 

This  assumption  is  supported  in  the  Defense  Acquisition 
Regulation  (35:1-337).  However,  not  all  analyses  support 
this  position.  Dr.  H.  G.  Hoehl  proposes  that  once  a  "com¬ 
petitive  contract  is  awarded,  the  contractor  essentially 
becomes  a  sole-source  supplier  (18:49]."  Mr.  David  Packard, 
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former  Deputy  Secretary  of  Defense,  holds  the  following 
views  on  the  virtues  of  contractor  competition: 

I  have  great  faith  in  the  competitive  free  enter¬ 
prise  system;  I  believe  it  to  be  generally  more  effi¬ 
cient  than  government  management.  But,  I  have  reluc¬ 
tantly  concluded  that  where  we  have  complex,  expensive 
military  systems,  and  where  only  one  customer  exists 
the  leveling  actions  of  the  free  market  economy  simply 
do  not  work  [3*/: 7]. 

It  is  clear  that  no  consensus  has  been  reached  on  this  point. 
Perhaps  investigation  into  the  competitive  versus  noncompe¬ 
titive  question  will  resolve  the  dispute  in  the  future. 

When  forming  a  Should  Cost  study  team,  one  of  the 
most  difficult  decisions  to  make  is  how  many  individuals  to 
include.  Kost  literature  encountered  advocates  the  tailor¬ 
ing  of  the  team’s  size  and  composition  to  the  task  at  hand. 
The  U.S.  Army  advocates  using  an  advance  team  to  scout  the 
contractor's  facility  to  determine  the  team's  requirements 
(34:3-5).  On  the  Air  Force  side,  Lt.  Gen.  Driessnack 
recently  stated  that  large  teams  are  not  needed  (8) .  Small 
teams  could,  he  said,  suffice  for  all  studies.  This  "mini" 
team  approach  employed  by  the  Air  Force  requires  fewer 
people  and  less  time.  However,  "experienced  Army  and  Navy 
Should  Cost  team  members  claim  that  the  Air  Force  is  not 
performing  Should  Cost  analysis,  but  rather  an  expanded 
pre-award  survey  [18:50] . " 

Industry  seems  to  hold  a  point  of  view  which  agrees 
with  the  Air  Force's.  A  study  by  R.  W.  Haight,  Lt.,  USN, 
found  contractors  generally  preferred  small  teams  made  up 
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of  "independent  consultants  [in  order  to]  remove  any  paro¬ 
chialism  that  would  exist  had  the  team  been  made  up  of  per¬ 
sonnel  from  the  activity  responsible  for  the  procurement 
[14:31]."  That  particular  study  also  indicated  that  a  team 
of  this  nature  would  foster  more  cooperation  between  the 
team  and  contractor  and  reduce  the  contractor's  support 
requirements.  Haight  clearly  indicated  that  the  contrac¬ 
tors  made  no  mention  of  the  fact  that  small  teams  are  easier 
to  manipulate  than  large  teams  (14:32). 

Perhaps  a  systems  approach  to  the  question  of  team 
size  would  be  helpful.  This  approach  identifies  the  desired 
output  in  terms  of  depth  of  analysis  and  scope  of  study. 

The  processes  used  to  derive  the  outputs  will  consist  of 
input  factors  chosen  with  respect  to  time,  personnel,  ex¬ 
pertise,  and  funds  available.  Inputs  are  also  needed  from 
previous  studies  of  similar  tasks  to  evaluate  the  composi¬ 
tion  of  teams  judged  "successful"  in  the  past. 

Another  point  to  consider  when  fielding  a  Should 
Cost  team  is  who  will  provide  the  personnel.  The  Government 
Accounting  Office  (GAO)  concluded  that  analyses  should  be 
"performed  by  procuring  activities  as  part  of  the  pre-award 
•analysis  [12:28]."  Another  study  concluded  that  the  GAO 
3hould  conduct  all  Should  Cost  studies  (7:48).  Still 
another  study  recommended  that  Air  Force  Should  Cost  respon¬ 
sibilities  be  assigned  to  a  single  office:  Air  Force  Con¬ 
tract  Management  Division  (36:124) .  Finally,  Lt.  Robert 
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Seiwart,  USA,  proposed  that  Should  Cost  studies  should  be 
conducted  by  an  outside  firm  or  by  a  joint  industry /govern¬ 
ment  team  (25:40),  These  differing  views  emphasize  the 
degree  of  uncertainty  in  the  application  of  Should  Cost, 
but  perhaps  there  is  no  one  best  agency  to  oversee  Should 
Cost.  The  need  to  tailor  a  Should  Cost  study  to  each  situa¬ 
tion  may  well  extend  to  the  assignment  of  responsibility 
for  the  study. 

Since  the  initial  Should  Cost  study  at  Pratt  & 
Whitney  in  1967,  numerous  additional  studies  have  been  per¬ 
formed.  Between  1973  and  1979,  the  Army  conducted  89 
Trudies,  the  Air  Force  37,  and  the  Navy  three  (one  of  which 
was  conducted  by  consultants) (18:50) .  Unfortunately,  few 
in-depth  studies  analyzing  the  costs  versus  the  benefits 
of  the  Should  Cost  technique  have  been  performed.  There 
seems  to  be  a  great  deal  of  discussion  and  very  little 
actual  empirical  analysis. 

One  particular  area  that  has  attracted  some  limited 
attention  is  the  cost  of  conducting  a  Should  Cost  study. 

One  of  the  most  expensive  studies  involved  an  investigation 
into  the  MK-48  torpedo  for  the  Navy  (7:19).  The  estimated 
price  tag  in  that  case  was  approximately  $4  million.  This 
illustrates  the  potential  high  costs  that  may  be  incurred 
in  performing  Should  Cost  analyses.  This  factor  should  be 
considered  in  light  of  potential  cost  savings  before  ini¬ 
tiating  a  study.  Joseph  R.  Weis,  in  a  research  report  for 
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the  Florida  Institute  of  Technology,  reviewed  30  Should 
Cost  studies  conducted  by  -the  Army  from  1973  to  1977.  The 
results  are  encouraging  for  proponents  of  the  technique. 
Costs  for  the  studies  ranged  from  .2  to  17.7  percent  of 
the  procurement  savings  attributed  to  Should  Cost.  Of  the 
30  Should  Cost  studies  reviewed,  the  most  expensive  cost 
the  Army  $201,672  to  perform.  The  resultant  savings  in  that 
study  were  $5.4  million  (37:15).  Moot  financiers  would 
agree  that  such  a  large  net  return  is  quite  impressive.  A 
summary  of  the  30  contracts  in  Weis'  study  shows  the  follow¬ 
ing  (17:217): 

Proposed  Total  Contract  Costs  $1,473.9  million 

Negotiated  Total  Contract  Costs  -1,239.8  million 

Total  Reduction  234.1  million 

Historical  Savings  -86.4  million 

Should  Cost  Savings  147.7  million 

Cost  of  Should  Cost  Studies  -1.6  million 

NET  SAVINGS  146.1  million 

The  historical  savings  were  obtained  from  comparison  of  pre¬ 
vious  procurements  of  the  same  type  with  each  Should  Cost 
study. 

Dr.  Hoehl  investigated  44  Army  contracts  negotiated 
between  1975  and  1979  for  which  Should  Cost  analyses  were 
performed.  He  found  a  difference  of  $691  million  between 
the  proposed  and  negotiated  costs.  The  costs  of  the  Should 
Cost  studies  was  $7.5  million.  No  data  were  obtained  to 
allow  comparison  of  these  savings  with  what  might  have  been 
realized  using  other  acquisition  techniques.  In  a  1974 
study  done  for  the  Defense  Systems  Management  School,  23 
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Should  Cost  efforts  conducted  by  the  U.S.  Army  were  ana¬ 
lyzed  (22:14)  .  A  total  reduction  of  $141.1  million  was 
achieved  in  negotiations.  Historical  data  indicated  that 
conventional  cost  analysis  techniques  used  on  acquisitions 
of  similar  size  and  type  would  have  resulted  in  savings  of 
$52.5  million.  Of  the  $141.1  million,  $88.6  million  was 
attributed  to  the  use  of  Should  Cost.  The  application  of 
Should  Cost  analysis  more  than  doubled  the  normal  cost 
reduction  achieved  in  negotiations. 

Dr.  Hoehl  reviewed  37  Should  Cost  studies  conducted 
by  the  Air  Force  between  1973  and  1979.  He  found  that  the 
Air  Force  saved  $565  million  during  contract  negotiations 
while  spending  only  $1.4  million  for  the  Should  Cost  studies 
(18:50-55).  In  the  Defense  Systems  Management  School  study 
cited  earlier,  five  Air  Force  Should  Cost  efforts  were 
analyzed.  A  total  cost  reduction  of  $131.1  million  was 
achieved  in  negotiations,  of  which  $65.3  million  was  attri¬ 
buted  to  Should  Cost  efforts.  Once  again,  more  than  half 
of  the  total  reduction  in  costs  realized  during  contract 
negotiations  resulted  from  the  application  of  Should  Cost. 

In  a  speech  at  the  Air  Force  Pricing  Conference  on 
13  July  1982,  Lt.  Gen.  H.  H.  Driessnack  presented  the  fol¬ 


lowing  data: 


Should  Other 

Cost  Techniques 


Number  of  contracts  12  15 

Initial  proposal  $613. 2M  $539. 2M 

Negotiated  price  512. OM  506. 4M 

Dollar  reduction  101. 2M  32. 8M 

Percent  reduction  16.5%  6.1% 
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The  percent  reductions  from  initial  proposal  to  final  nego¬ 
tiated  price  for  those  contracts  in  which  Should  Cost  was 
used  are  almost  three  times  as  high  as  the  percentages  for 
non-Should  Cost  contracts.  Unfortunately,  no  data  were 
available  as  to  when  the  studies  were  conducted.  At  least 
on  the  surface,  the  Should  Cost  studies  appear  to  be  provid¬ 
ing  a  more  effective  means  of  negotiating  cost  reductions. 

In  1976,  a  statistical  analysis  was  conducted  on  30 
U.S.  Army  Should  Cost  studies.  The  researchers  attempted 
to  determine  whether  or  not  a  relationship  existed  between 
the  prices  initially  proposed  by  contractors  and  the  size 
of  the  difference  between  the  initial  proposal  and  the  final 
negotiated  price.  The  coefficient  of  determination  (R- 
squared)  was  calculated  to  be  .9686.  (A  value  of  1.0  for 
R-squared  indicates  a  perfect  linear  correlation.)  When 
the  initial  proposals  were  compared  with  the  corresponding 
differences  between  proposed  price  and  Should  Cost  estimate, 
R-squared  was  found  to  be  .9403  (9:36).  No  significant 
relationships  were  found,  however,  when  the  initial  pro¬ 
posals  were  compared  with  the  differences  between  proposed 
price  and  negotiated  price  and  between  proposed  price  and 
Should  Cost  estimate  as  a  percentage  of  the  initial  propos¬ 
al  (9:39).  (Values  of  R-squared  were  0.0087  and  0.0161 
respectively.)  The  study  revealed  that  the  percent  reduc¬ 
tions  "seem  to  have  been  relatively  constant  within  a  cer¬ 
tain  range  [9:26]."  The  constant  percentage  would  account 


for  the  linear  relationship  noted  above.  As  the  contractor 
proposals  increased,  so  too  did  the  absolute  dollar  differ¬ 
ence  between  the  proposals  and  the  Should  Cost  estimates 
and  the  negotiated  prices.  It  should  be  pointed  out  that 
the  statistical  analysis  did  not  try  to  consider  the  money 
that  might  have  been  saved  had  Should  Cost  not  been  used. 

Lt.  Col.  William  E.  Schaefer  and  Major  Roy  F. 
Birkhead  conducted  another  study  at  the  Air  Force  Institute, 
of  Technology  in  1975.  They  compared  four  completed  con¬ 
tracts  negotiated  using  Should  Cost  with  four  previous  year 
buys  for  the  same  weapon  systems  where  conventional  cost 
analysis  techniques  were  used.  Schaefer  and  Birkhead 
attempted  to  determine  if  there  was  any  difference  between 
the  actual  cost  outcomes  on  contracts  in  the  two  groups 
(24 ill) .  One  of  their  five  tests  compared  the  proposed 
cost  to  negotiated  cost  for  the  Should  Cost  and  non-Should 
Cost  contracts.  They  found  that,  at  the  .05  significance 
level,  the  null  hypothesis  of  no  difference  could  not  be 
rejected  (24:57).  No  significant  difference  could  be  found 
between  the  percent  changes  from  proposed  to  negotiated 
costs.  While  interesting,  these  results  are  somewhat  sus¬ 
pect  due  to  the  sampling  methods  and  statistical  techniques 
used. 

In  performing  their  analysis,  Schaefer  and  Birkhead 
obtained  a  convenience  sanqple  containing  four  Should  Cost 
and  four  non-Should  Cost  contracts.  They  then  employed  the 
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paired  t  Statistic  to  test  their  hypothesis.  The  use  of  a 
parametric  test,  such  as  the  t  test,  requires  that  several 
basic  assumptions  be  made.  For  instance,  the  populations 
must  be  assumed  to  be  normally  distributed,  or  the  samples 
must  be  sufficiently  large  random  samples  to  apply  the 
Central  Limit  Theorem.  The  populations  from  which  Schaefer 
and  Birkhead  drew  their  samples  may  be  normally  distributed, 
but  there  is  no  real  basis  for  assuming  so.  Their  samples 
certainly  were  not  random,  and  a  sample  size  of  four  is 
not  large  enough  to  justify  invoking  the  Central  Limit 
Theorem.  Therefore,  the  small  samples,  the  way  in  which 
they  were  chosen,  and  the  fact  that  a  parametric  test  was 
used  all  call  question  to  the  validity  of  the  results  ob¬ 
tained  by  Schaefer  and  Birkhead.  Their  conclusions  should, 
by  no  means,  be  discarded.  However,  they  should  be  treated 
cautiously. 

Summary  of  Past  Studies 

It  is  evident  that  much  more  work  remains  to  be  done. 
The  number  of  studies  concerning  costs  and  savings  is  scant, 
and  there  is  substantial  disagreement.  In  the  analysis  of 
Schaefer  and  Birkhead,  the  small  sample  size  could  cast 
doubt  on  the  validity  of  the  results.  Additionally,  no 
studies  were  encountered  that  were  concerned  with  the  ef¬ 
fects  different  types  of  contracts  might  have  on  the  effec¬ 
tiveness  of  Should  Cost.  For  example,  do  contracts  such  as 
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Firm  Fixed  Price  and  Fixed  Price  Incentive  Firm  show  dif¬ 
ferent  savings  than  other,  cost  type  contracts.  There  is 
disagreement  over  whether  Should  Cost  is  applicable  to  sole 
source  and/or  competitive  procurements.  Investigation  of 
the  effects  of  the  acquisition  phase  in  which  Should  Cost 
analyses  are  conducted  is  also  needed.  Finally,  there  is 
no  concurrence  among  analysts  as  to  the  agency  which  should 
conduct  Should  Cost  studies.  Without  further  investigation 
into  these  areas,  the  real  worth  of  Should  Cost  cannot  be 
determined . 

Problem  Statement  and  Objective 

Given  the  Should  Cost  findings  mentioned  in  the  pre 
ceding  literature  review,  there  remains  some  doubt  as  to 
whether  there  are  actual  cost  savings  realized  by  the  DOD 
through  the  application  of  Should  Cost.  The  objective  of 
this  analysis  of  Should  Cost  contract  data  was  to  determine 
if  alleged  procurement  savings  are  significantly  greater 
than  savings  realized  through  the  use  of  other  acquisition 
techniques. 


Research  Hypotheses 

The  following  three  research  hypotheses  were  inves¬ 
tigated  in  this  study: 


[ 
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There  is  no  significant  difference  in  the  percent 
cost  reduction  during  negotiations  between  Air  Force  con¬ 
tracts  involving  Should  Cost  and  those  where  Should  Cost  is 
not  used. 


There  is  no  significant  difference  in  the  percent 
cost  reduction  during  negotiations  between  Army  contracts 
involving  Should  Cost  and  those  where  Should  Cost  is  not 
used. 


There  is  no  significant  difference  in  the  percent, 
cost  reductions  during  negotiations  between  Air  Force  and 
Army  acquisitions  that  employ  the  Should  Cost  technique. 


CHAPTER  2 


METHODOLOGY 

This  chapter  contains  an  explanation  of  the  data 
collection  and  statistical  test  techniques  used  in  this 
analysis . 

Data  Collection 

The  first  two  hypotheses  stated  in  Chapter  1  are 
concerned  with  determining  whether  or  not  there  is  a  sig¬ 
nificant  difference  in  percent  cost  reductions  during  nego¬ 
tiations  for  Should  Cost  and  non-Should  Cost  contracts. 

The  statistical  tests  employed  here  were  the  Wilcoxon 
Signed  Rank  Test  for  comparing  paired  samples  from  two 
populations,  and  the  Mann -Whitney  Test  for  comparing  inde¬ 
pendent  samples  from  two  populations.  The  third  hypothesis 
involves  comparing  Should  Cost  contracts  for  the  Air  Force 
with  those  for  the  Army  to  determine  if  either  of  the  two 
diverse  applications  of  the  technique  provide  better 
results.  The  Mann-Whitney  Test  was  again  applied  here. 

The  reasons  for  using  these  particular  tests  will  be  dis¬ 
cussed  later  in  this  chapter.  To  perform  the  tests,  sam¬ 
ples  must  be  drawn  from  two  populations.  The  populations 
from  which  samples  were  drawn  in  this  analysis  differ  for 
each  hypothesis  test. 


For  Hypothesis  1,  one  population  is  made  up  of  all 
weapon  system  acquisition  contracts  entered  into  by  the  Air 
Force  since  the  Should  Cost  technique  was  first  used.  The 
second  population  includes  only  those  contracts  in  which 
Should  Cost  was  employed.  The  populations  for  Hypothesis  2 
are  the  same  as  those  for  Hypothesis  1  except  that  they 
contain  only  Army  procurement  contracts.  For  Hypothesis  3, 
one  population  contains  Air  Force  contracts  for  which  Should 
Cost  was  used,  and  the  other  contains  Army  contracts  where 
Should  Cost  was  used. 

DOD  contracts  generally  fall  into  one  of  two  cate¬ 
gories:  fixed-price  and  cost  reimbursement.  In  a  fixed- 
price  agreement,  the  cost  of  a  particular  system  is  agreed 
upon  by  both  parties.  In  addition,  some  fixed  profit  amount 
or  some  share  arrangement  for  any  reductions  in  cost  is 
negotiated.  This  type  of  contract  is  usually  used  when 
there  is  little  technical  risk  involved;  when  production 
methods  and  design  have  been  established.  The  types  of 
fixed-price  contracts  dealt  with  in  this  study  are  firm 
fixed  price  (FFP)  and  fixed  price  incentive  firm  (FPIF) . 

A  FFP  contract  is: 

An  agreement  to  pay  a  specified  price  when  the 
items  specified  in  the  contract  have  been  delivered  and 
accepted.  The  price  for  the  original  work  is  not  ad¬ 
justed  afterward,  regardless  of  the  contractor's  actual 
cost  experience  [35:131] . 

There  are,  of  course.,  exceptions  for  changes  in  accordance 
with  the  change  clause  and  other  applicable  contract 


clauses.  A  FPIF  contract  is  used  where  contractor  risk 
cannot  be  reduced  to  a  level  acceptable  for  use  of  a  FFP 
contract  (35:132).  The  basic  elements  of  the  FPIF  contract 
are  target  cost,  target  profit,  ceiling  price.,  and  share 
ratio.  The  share  ratio  specifies  the  portion  of  any  cost 
increase  above  target  cost  which  the  government  will  pay 
or  the  portion  of  any  cost  decrease  the  government  will 
receive. 

Cost  reimbursement  contracts  are  used  in  situations 
characterized  by  high  risk.  For  example,  this  type  of  con¬ 
tract  is  often  used  in  the  area  of  research  and  development. 
Cost  reimbursement  contracts  may  be  used  when  specifica¬ 
tions  are  incomplete  or  when  there  is  doubt  about  whether 
the  contract  can  be  successfully  accomplished  (35:134). 

The  common  factor  here  is  a  high  degree  of  risk.  Contract 
costs  cannot  be  accurately  estimated.  The  contractor  in 
this  case  is  reimbursed  for  all  costs  incurred.  In  addi¬ 
tion,  the  government  pays  some  fee  to  the  contractor.  The 
fee  may  be  a  negotiated  target  amount,  or  may  be  variable 
according  to  some  incentive  schedule.  Under  the  variable, 
or  incentive  plan,  the  contractor  receives  a  larger  fee 
when  costs  are  reduced. 

The  inherent  differences  between  the  two  types  of 
contracts,  fixed-price  and  cost  reimbursement,  make  it 
likely  that  contract  type  will  have  some  effect  on  price 
negotiations.  Therefore,  it  is  desirable  to  separate  con- 
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tract  data  bared  on  the  type  of  contract  involved.  Unfor¬ 
tunately,  Should  Cost  is  usually  applied  only  to  contracts 
for  production  of  weapon  systems  which  have  already  been 
manufactured  in  the  past  and  for  which  design  and  production 
processes  have  already  been  established.  As  has  been 
pointed  out,  these  procurements  usually  lend  themselves  to 
the  use  of  fixed-price  contracts.  In  fact,  all  of  the  data 
obtained  involved  either  FFP  or  FPIF  contracts.  Therefore, 
no  basis  for  comparing  the  relative  benefits  of  Should  Cost 
'for  fixed-price  versus  cost  reimbursement  contracts  was 
available  to  the  researchers. 

The  data  collected  consist  of  the  contractor's  pro¬ 
posed  price  (or  target  price)  and  the  negotiated  price  (or 
target,  orice) .  Contractual  changes  affecting  both  price 
and  specif ication  w ill  not  be  taken  into  account.  The  con¬ 
cern  of  this  research  is  with  the  price  reduction  during 
initial  award  negotiations,  and  how  the  use  of  Should  Cost 
affects  it.  There  may  be,  and  usually  are,  many  contract 
changes  during  the  life  of  weapons  systems.  These  changes 
are  normally  accompanied  by  increases  in  the  cost  of  the 
systems  involved.  It  is  not  the  purpose  of  this  study  to 
attempt  to  determine  the  effects  of  Should  Cost  on  contract 
changes  in  out  years.  For  this  reason,  design  and  price  • 
changes  made  after  the  contracts  were  negotiated  are  not 
considered. 
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The  data  gathered  for  this,  analysis  consists  of  a 
convenience  sample.  A  convenience  sample  is  simply  a 
sample  containing  data  which  is  available  to  the  researcher. 
The  nature  of  a  convenience  sample  generally  renders  it 
unrepresentative  of  the  population  from  which  it  is  drawn. 
Therefore,  the  results  may  be  biased  (16:256).  Still,  such 
a  sample  is  not  without  merit.  It  may  provide  evidence  so 
overwhelming  it  is  unnecessary  to  use  more  sophisticated 
sampling  techniques  (23:177). 

A  convenience  sample  was  used  in  this  study  because 
of  the  general  lack  of  historical  contract  data.  Neither 
the  Air  Force  nor  the  Army  has  established  a  centralized 
repository  for  data  on  Should  Cost  contracts.  The  search 
for  data  was  restricted  to  the  U.S.  Army  Materiel  Develop¬ 
ment  and  Readiness  Command  (DARCOM)  and  its  subcommands, 
and  the  U.S.  Air  Force  Systems  Command  and  its  product 
divisions.  These  entities  are  the  research  and  development 
and  procurement  mechanisms,  and  the  center  of  nearly  all 
Should  Cost  activities  in  their  respective  branches.  In 
many  cases,  the  organizations  queried  either  did  not  main¬ 
tain  or  were  reluctant  to  search  for  the  type  of  data 
needed  to  perform  this  analysis. 

The  search  for  data  was  hampered  by  another  factor 
which  necessitated  the  use  of  a  convenience  sample.  In 


order  to  perform  the  Wilcoxon  Test  for  Hypotheses  1  and  2, 
samples  containing  matched  pairs  of  observations  were 
needed.  To  provide  for  some  natural  pairing  of  observed 
values,  the  researchers  decided  to  arrange  the  data  such 
that  each  contained  one  Should  Cost  contract  and  one  non- 
Should  Cost  contract  for  different  production  lots  of  the 
same  weapon  system.  For  an  item  to  provide  useful  data, 
therefore,  it  had  to  be  characterized  by  at  least  one 
Should  Cost  acquisition  and  at  least  one  "conventional" 
acquisition.  In  the  Army,  a  Should  Cost  analysis  is  per¬ 
formed  on  items  "lacking  adequate  price  competition  and 
exceeding  $25  million  in  contract  price  [32:2] ."  This 
implies  that  an  item  must  have  reverted  from  a  competitive 
procurement  to  a  sole- source  procurement  or  vice-versa  in 
order  to  provide  the  required  matched  pair.  This  does  not 
normally  occur  for  items  procured  in  DOD.  Therefore,  the 
data  available  was  somewhat  limited.  This  fact  may  call 
question  to  the  generalizability  of  any  results  obtained 
in  this  analysis. 

Ideally,  the  data  used  in  this  study  should  be  dif- 
.  ferentiated  only  by  the  presence  or  absence  of  a  Should 
Cost  analysis  for  Hypotheses  1  and  2,  or  by  the  method  of 
analysis,  either  Air  Force  or  Army,  for  Hypothesis  3.  As 
the  researchers  discovered,  however,  this  is  rarely  the 
case.  Many  factors  come  into  play  during  the  contract 
negotiation  phase,  and  these  factors  often  change  from  one 
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contract  to  the  next.  Some  of  the  factors  affecting  nego¬ 
tiations  include: 

-  contractor 

-  the  people  involved  in  negotiations 

-  number  of  units  acquired 

-  modifications  distinguishing  one  model  from 
another 

-  changes  in  production  methods 

-  upgrading  production  facilities 

-  method  of  overhead  cost  allocation 

-  increased  experience  level  of  production  per¬ 
sonnel  or  personnel  turnover 

-  economic  conditions  influencing  costs  of 
materials 

-  availability  of  substitute  items 

Some  of  the  factors  above  can  be  controlled.  By  comparing 
d*ta  from  different  production  lots  of  the  same  weapon  sys¬ 
tem,  the  problem  of  differing  contractors  can  be  eliminated. 
In  addition,  the  same  people  are  often  involved  in  negotia¬ 
tions,  and  the  method  of  overhead  allocation  usually 
remains  the  same.  Where  there  was  a  choice,  contracts  were 
chosen  such  that  the  quantities  purchased  were  as  close  as 
possible.  Since  Should  Cost  concepts  have  generally  only 
been  applied  to  high  cost  contracts  for  established  systems, 
there  are  not  generally  a  great  many  substitute  items  avail¬ 
able.  The  fact  that  the  data  used  in  this  study  is  made  up 
of  the  percent  reduction  in  price  during  negotiations 
should  help  control  for  any  change  in  economic  conditions. 
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The  other  factors  listed  above  cannot  be  controlled  in  a 
study  employing  a  convenience  sample  as  this  one  does. 
They  therefore  must  be  considered  moderating  variables  in 
this  analysis. 


Statistical  Analysis  Method 


As  was  stated  previously,  the  Wilcoxon  Signed  Rank 
Test  and  the  Mann-Whitney  Test  were  used  in  this  study. 
These  tests  come  from  a  relatively  new  class  of  statistical 
methods  called  nonpar ame trie,  or  distribution-free  methods. 
There  are  three  major  reasons  in  this  case  for  using  non- 
parametric  techniques.  First,  the  assumption  cannot  be 
made  that  the  samples  are  drawn  from  normal  populations. 
This  is  so  for  a  number  of  reasons: 

1.  The  populations  include  a  variety  of 
weapon  systems  representing  a  non- 
homogeneous  group. 

2.  Contracts  are  for  varying  quantities. 

3.  Defense  contractors  are  not  a  homo¬ 
geneous  group. 

4.  The  use  of  Should  Cost  studies  has 
been  limited  to  high  cost  contracts 
selected  at  the  discretion  of  the 
procuring  organization.  (This  has 
recently  changed  in  the  Army  where 
Should  Cost  is  now  mandated  for  con¬ 
tracts  over  a  certain  dollar  thresh¬ 
old.) 

The  second  reason  for  the  use  of  a  nonparametric 
test  is  the  method  of  sample  collection  and  the  size  of  the 
samples.  The  Central  Limit  Theorem  says  that  if  random 
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samples  of  sufficient  size  are  used,  they  can  be  assumed  to 
be  normally  distributed  regardless  of  the  distribution  of 
the  parent  population.  The  samples  in  this  study,  however, 
are  convenience  samples,  and  are  not  generally  large  enough 
to  invoke  the  Central  Limit  Theorem. 

Third,  the  variances  for  the  populations  cannot  be 
assumed  to  be  equal.  This  is  true  for  basically  the  same 
reasons  as  those  listed  above.  Snedecor  and  Cochran  indi¬ 
cate  that  "when  samples  come  from  populations  of  different 
types",  their  variances  should  be  assumed  to  be  different 
(27:96).  The  researchers  felt  that  the  above  factors  war¬ 
ranted  the  use  of  nonpar ametric  methods  in  analyzing  the 
data.  While  these  techniques  may  require  more  care  in 
interpreting  results,  the  more  robust  nature  of  the  distri¬ 
bution-free  tests  makes  them  the  logical  choice  in  this 
study . 

According  to  a  text  written  by  Larry  E.  Richards 
and  Jerry  J.  LaCava,  three  conditions  should  be  satisfied 
before  the  Wilcoxon  test  is  used  (23:348).  First,  there 
should  be  a  natural  pairing  of  observations.  Second,  the 
data  should  be  at  least  interval  level  data;  and  third, 
data  should  be  produced  sudh  that  tied  values  of  the  paired 
differences  are  not  very  likely.  An  attempt  was  made  to 
collect  data  in  such  a  way  as  to  allow  the  formation  of 
matched  pairs,  each  containing  a  Should  Cost  and  a  non- 
Should  Cost  contract  for  the  same  weapon  system.  This,  it 


was  hoped,  would  provide  some  natural  pairing  of  the  obser¬ 
vations.  The  data  are  in  the  form  of  percentages-- speci¬ 
fically,  the  percent  differences  between  proposed  and  nego¬ 
tiated  prices.  It  is  therefore  considered  to  be  ratio 
level  data.  Because  the  data  was  derived  as  a  percentage, 
ties  are  very  unlikely. 

As  stated  before,  in  using  the  Wilcoxon  Signed  Rank 
Test,  the  data  are  arranged  in  matched  pairs.  Each  pair 
contains  one  contract  in  which  Should  Cost  was  used  and 
one  in  which  it  was  not.  Looking  then  at  the  percent  charige 
from  contractor  proposal  to  negotiated  price,  the  difference 
between  the  percentages  for  each  pair  is  computed.  The 
absolute  values  of  these  differences  are  then  ranked  from 
lowest  to  highest.  The  test  statistic  becomes  the  smaller 
of  the  rank  sums  of  either  the  positive  or  the  negative 
differences. 

The  sums  of  the  ranks  for  the  positive  and  negative 
differences  are  obtained  separately.  These  sums  form  the 
basis  for  testing  the  null  hypothesis  Ho:  Rank  Sum  (+)  = 
Rank  Siam  (-) .  The  null  hypothesis  is  that  the  populations 
from  which  the  two  samples  are  drawn  are  identical.  If 
this  is  indeed  the  case,  the  population  positive  and  nega¬ 
tive  differences  should  be  symmetrically  distributed  about 
a  mean  of  zero  (15:525)  .  If  the  smaller  of  the  two  rank 
sums,  conventionally  known  as  Wilcoxon 's  t  statistic,  is 
equal  to  or  less  than  the  critical  value  of  t,  the  null 


Sfc**£*illE*3Ss <k,Z  *, a  ViV*‘i* 


■  vt  VvVA  A  .%  *%  VN  \' 


t'*  #r*  «*x  o6  ir*.y*  ^  -jsj 


hypothesis  can  be  rejected.  In  testing  Hypotheses  1  and  2 
in  this  analysis#  if  the  rank  sum  of  the  positive  differ¬ 
ences  is  equal  to  or  less  than  the  critical  value#  it  can 
be  concluded  that  there  is  some  significant  increase  in 
percent  cost  reductions  during  negotiations  when  Should 
Cost  is  applied.  If#  however#  the  null  hypothesis  cannot 
be  rejected#  the  percent  cost  reductions  must  be  assumed 
tc  be  equal  regardless  of  whether  Shou3.d  Cost  is  used  or 
not. 

As  a  further  test  of  Hypothesis  1,  and  in  analyzing 
the  data  for  Hypothesis  3#  the  researchers  employed  the 
Mann-Whitney  Test.  Much  of  the  data  obtained  from  the  Air 
Force  could  not  be  used  in  the  Wilcoxon  Test.  Therefore# 
the  researchers  decided  to  perform  a  second  test,  the  Mann- 
Whitney,  which  would  allow  inclusion  of  the  entire  set  of 
available  data.  In  addition,  it  was  decided  that  it  would 
be  difficult  or  impossible  to  arrange  data  in  matched  pairs 
in  order  to  perform  a  Wilcoxon  Signed  Rank  Test  for  Hypoth¬ 
esis  3.  The  inherent  differences  in  the  types  of  weapons 
procured  for  Army  and  Air  Force  use  are  too  great  to  per¬ 
mit  any  natural  pairing  of  observations.  Therefore#  the 
Mann-Whitney  Test  is  used  here.  The  first  step  in  the 
Mann-Whitney  Test  is  to  rank  all  observations  from  the  two 
independent  samples  from  smallest  to  largest.  Values  are 
assigned  to  each  observation  according  to  its  rank,  and 
rank  sums  are  computed  for  both  samples.  These  rank  sums# 
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termed  Ta  and  Tb  form  the  basis  for  computing  the  test 
statistic,  U.  If  the  populations  are  identical,  the  rank 
sums  for  two  samples  will  tend  to  be  equal  (21:783).  How¬ 
ever,  if  some  difference  exists  between  the  two  populations 
there  will  be  a  tendency  for  one  of  the  rank  sums  to  be 
larger  than  the  other.  Two  values  for  the  test  statistic 
are  computed  in  the  following  manner: 

Ua  -  n,n„  +  ni(ni+1)  _  Ta 

1  2  - g - 

Ub  a  n,n„  +  n2(n2+1)  -  Tb 
Li  2 

Large  values  for  the  rank  sums  lead  to  small  corresponding 
values  for  the  U  statistic.  Therefore,  to  determine  if 
population  A  is  shifted  to  the  right  of  population  B,  which 
implies  that  population  A  contains  generally  larger  values, 
Ua  would  be  compared  to  some  specified  value,  Uo.  If  Ua  is 
less  than  Uo,  the  null  hypothesis  of  no  difference  can  be 
rejected;  and  it  can  be  concluded  that  population  A  is 
indeed  shifted  to  the  right  of  B.  If  Ua  is  greater  than  Uo 
and  Ub  tests  out  similarly,  the  null  hypothesis  cannot  be 
rejected.  The  populations  must  be  assumed  to  be  identical. 

While  the  two  nonpar ametric  methods  employed  here 
are  relatively  easy  to  apply,  and  while  they  do  not  require 
the  limiting  assumptions  necessary  for  parametric  tests, 
there  are  some  disadvantages  to  their  use.  Because  the 
tests  involve  the  use  of  ordering  or  ranking  as  opposed  to 


the  actual  numerical  values  of  the  observations,  some  infor¬ 
mation  may  be  lost  (15:519).  While  it  may  be  possible  to 
determine  if  one  population  is  shifted  to  the  left  or  right 
of  another,  the  magnitude  of  the  shift  is  not  known.  For 
this  reason,  it  was  not  possible  in  this  analysis  to  take 
the  incremental  funds  expended  for  Should  Cost  studies  into 
account.  The  researchers  decided,  however,  that  the  nature 
of  the  populations  and  the  method  of  sample  collection  man¬ 
dated  the  use  of  a  distribution-free  technique. 


CHAPTER  3 


DATA  ANALYSIS 

This  chapter  presents  the  data  obtained  and  the 
statistical  tests  performed  for  each  of  the  three  research 
hypotheses  stated  in  Chapter  1.  Part  1  concerns  Air  Force 
data  used  to  test  the  validity  of  Hypothesis  1.  Part  2 
presents  Army  data  for  testing  Hypothesis  2,  and  Part  3 
is  concerned  with  the  data  from  both  the  Air  Force  and  the 
Army  used  to  check  the  validity,  of  Hypothesis  3. 


>othesis  1 


There  is  no  significant  difference  in  the  percent 
cost  reduction  during  negotiations  between  Air  Force 
contracts  involving  Should  Cost  and  those  where  Should 
Cost  is  not  used. 

All  data  collected  on  Air  Force  Should  Cost  procure¬ 
ments  were  obtained  from  the  Pricing  Division  of  Aeronauti¬ 
cal  Systems  Division,  Air  Force  Systems  Command.  Non- 
Should  Cost  contract  data  was  obtained  from  the  Aeronautical 
Systems  Division  Cost  Library.  Table  3-1  is  a  compilation 
of  all  data  obtained  on  Air  Force  acquisitions.  As  stated 
in  Chapter  2,  this  data  lends  itself  to  two  basic  types  of 
statistical  tests:  a  paired  test  and  an  unpaired  test. 

The  two  tests  used  were  the  Wilcoxon  Signed  Rank  Test  and 
the  Mann-Whitney  Test.  The  Wilcoxon  Test  was  performed  in 
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an  attempt  to  control  for  external  factors  which  might 
cause  some  correlation  between  observations.  The  Mann- 
Whitney  Test  was  also  performed  because  arranging  the  data 
in  matched  pairs  for  the  Wilcoxon  Test  meant  that  many 
observations  had  to  be  omitted. 

Table  3-2  shows  the  data  used  for  the  Wilcoxon 
Signed  Rank  Test.  This  table  contains  a  subset  of  the  data 
presented  in  Table  3-1.  Observations  have  been  selected 
such  that  matched  pairs  can  be  formed.  Each  matched  pair 
contains  two  contracts  for  the  same  weapon  system,  from  the 
same  contractor,  with  contract  costs  matched  as  closely  as 
possible.  Once  these  matched  pairs  are  formed,  the  Wilcoxon 
Test  can  be  performed  as  shown  below. 

Ho:  Populations  are  identical 

Ha:  One  population  is  shifted  to  the  right 
or  left  of  the  other. 

Reject  Ho  if  the  rank  sum(+'  or  rank  sum(-) 
is  less  than  or  equal  to  3  * 

♦Critical  values  for  each  of  the  tests  per¬ 
formed  are  taken  from  H.R.  Neave's  Statis¬ 
tics  Tables.  All  are  based  on  a  two-tailed 
test  with  a  .05  significance  level. 

The  computations  needed  to  perform  the  Wilcoxon  .Test  are 
shown  in  tabular  form  below: 
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TABLF  3-3 


COMPUTATIONS  FOR  WILCOXON  TEST 

%  Reduction 

Rank 

Should  Cost  Non-Should  Cost 

Difference 

ill 

1 

13.93 

8.83 

+  5.10 

4 

2 

15.04 

10.92 

+  4.12 

3 

3 

14.46 

21.39 

-  6.93 

7 

4 

7.16 

9.03 

-  1.87 

1 

5 

13.37 

7.64 

+  5.73 

5 

6 

13.50 

7.22 

+  6.28 

6 

7 

7.73 

5.39 

+  2.34 

2 

8 

5.52 

16.59 

-11.07 

8 

Sum 

20 

16 

The  reader 

will  note  that 

the  lower  of 

the 

two  rank 

sums — that  for  the 

negative  differences — is  greater 

than 

the 

critical  value 

of  3.  Therefore,  based  on 

this 

sample , 

the  null  hypothesis  cannot  be  rejected.  It  cannot  be  con¬ 
cluded  that  there  is  a  significant  difference  between  the 
two  populations.  However,  as  Table  3-3  indicates,  the  rank 
sum  for  the  negative  differences  is  affected  significantly 
by  two  observations.  Observations  3  and  8  both  show  per¬ 
cent  cost  reductions  for  the  non-Should  Cost  contracts 
r  which  may  be  unusually  high:  21.39  and  16.59  percent, 

respectively.  Since  the  data  presented  here  represent  a 
small  convenience  sample,  there  is  a  strong  possibility 
that  the  distribution  of  the  samples  may  not  be  represen¬ 
tative  of  their  populations.  As  an  additional  check,  the 
entire  set  of  data  shown  in  Table  3-1  was  used  to  perform 
a  Mann -Whitney  Test. 


Ho:  Populations  are  identical 

Ha:  One  population  is  shifted  to  the  right 
or  left  of  the  other. 


Reject  Ho  if  Ua  or  Ub  are  less  than  or  equal 
to  106. 

The  test  statistic  for  the  Should  Cost  contracts,  Ua,  and 
that  for  the  non-Should  Cost  contracts,  ub,  are  calculated 
as  follows: 

Ua  -  (15)  (23)  +  -  332 

-  133 

Ub  *  (15)  (23)  +  -  409 

»  212 

Again,  neither  of  the  two  test  statistics  falls  below  the 
critical  value.  As  was  the  case  in  the  Wilcoxon  Test,  the 
null  hypothesis  cannot  be  rejected.  There  is  no  proof, 
based  on  the  sample  collected  for  this  treatise,  that  there 
is  a  significant  difference  between  the  percent  price  reduc¬ 
tions  during  negotiations  for  Air  Force  acquisitions  in 
which  Should  Cost  was  employed  and  those  in  which  it  was 
not. 
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TABLE  3-4 


AIR  FORCE  DATA  FOR  MANN-WHITNEY 


Should  Cost 

Non-Should 

Cost 

%  Reduction 

Rank 

%  Reduction 

Rank 

8.42 

17 

8.83 

18 

13.93 

29 

0.54 

5 

15.16 

32 

0.02 

2 

17.26 

34 

10.92 

24 

15.04 

31 

11.07 

25 

14.46 

30 

21.39 

38 

6.43 

11 

10.28 

22 

7.16 

12 

20.95 

37 

12.52 

26 

18.27 

35 

13.37 

27 

10.45 

23 

13.50 

28 

0.07 

3 

/.  73 

16 

9.03 

21 

5.92 

10 

7.52 

14 

9.00 

20 

7.64 

15 

5.52 

9 

7.22 

13  ' 

0.42 

4 

n  =  15 

332 

5.39 

8 

19.51 

36 

8.86 

19 

3.26 

6 

4.81 

7 

16.59 

33 

0.01 

1 

n  *  23 

409 
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Hypothesis  2 


There  is  no  significant  difference  in  the  percent 
cost  reduction  during  negotiations  between  Army  con¬ 
tracts  involving  Should  Cost  and  those  where  Should 
Cost  is  not  used. 

In  testing  this  hypothesis,  the  researchers  were  not 
as  fortunate  in  obtaining  data  as  was  the  case  with  the  data 
in  Part  1.  In  an  effort  to  collect  the  required  information, 
DARCOM  Headquarters  and  each  of  its  subcommands  were  con¬ 
tacted.  Requests  for  data  were  made  to  the  Should  Cost 
coordinator  of  each  organization.  The  data  collected  are 
presented  in  Table  3-5.  These  data  were  obtained  from  the 


following  sources: 

Weapon  System 

Helicopter  Engine 
Helicopter  A 

Helicopter  B 

Tow  Missile 


Source 

Troop  Support  and  Aviation  Materiel 
Readiness  Command 

DARCOM  Headquarters 

Missile  Command 


Most  of  the  remaining  subcommands  of  DARCOM  did  not  have 
the  requested  information.  One  office  that  indicated  it 
did  have  the  data  considered  it  proprietary  contractor  in¬ 
formation  and  would  not  release  it. 

As  Table  3-5  reflects,  data  collected  included  only 
six  Should  Cost  contracts  and  seven  non-Should  Cost  con¬ 
tracts.  These  contracts  involved  the  acquisition  of  only 
four  different  weapons  systems.  Considering  this  and  the 
method  of  sample  collection,  the  researchers  decided  that 
any  statistical  test  performed  on  these  data  would  be  incon- 
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elusive  at  best;  at  worst,  misleading.  It  was  decided, 
therefore,  that  a  qualitative  assessment  of  the  data  might 
-be  the  only  viable  option. 

Looking  at  Table  3-5,  it  becomes  apparent  that  the 
percent  price  reductions  for  Should  Cost  contracts  are 
generally  higher  than  those  for  non-Should  Cost  contract?. 
In  fact,  all  but  one  of  the  contracts  involving  the  use  of 
Should  Cost  show  higher  percent  reductions  than  12.8  per¬ 
cent,  which  is  the  highest  value  for  the  non-Should  Cost 
contracts.  Additionally,  when  contracts  for  like  commodi¬ 
ties  are  compared,  all  Should  Cost  contracts  show  higher 
.  percent  reductions.  It  would  certainly  be  improper  to 

assert  that  these  small  samples  are  representative  of  their 
respective  populations,  and  a  qualitative  analysis  of  the 
data  may  not  be  statistically  rigorous.  However,  it  does 
appear  here  that  Should  Cost  may  be  valuable  as  a  tool  in 
negotiating  weapon  system  procurements  in  the  Army. 

Hypothesis  3 

There  is  no  significant  difference  in  the  percent 
cost  reductions  during  negotiations  between  Air  Force 
and  Army  acquisitions  that  employ  the  Should  Cost  tech¬ 
nique  . 

In  testing  Hypothesis  3 ,  the  researchers  used  data 
on  Should  Cost  contracts  from  the  Air  Force  and  the  Army  to 
try  to  detect  any  possible  difference.  As  was  the  case  in 
Part  2'  above,  the  small  sample  size  for  the  Army  data  was 


47 


sssssbixxxspxsbcx^^p  rp  -pm rp\ v>.  i".^,  n  *:  .,*x*"C 


•.~'.’rK'  VLV.Y 


\V.VLVl^TVtVF 


a  limiting  factor.  The  researchers  decided,  however,  that 
with  six  data  points  for  the  Army  and  15  for  the  Air  Force, 
the  Mann-Whitney  Test  could  be  performed  with  some  degree 
of  reliability.  Data  for  this  test  are  drawn  from  Tables 
3-1  and  3-5  and  summarized  in  Table  3-6.  These  data  are 
used  to  conduct  a  Mann-Whitney  Test  in  the  following  manner 

Ho:  Populations  are  identical 

Ha:  One  population  is  shifted  to  the  right 
or  left  of  the  other. 

Reject  Ho  if  Ua  or  Ub  are  less  than  or  equal 
to  19. 

The  test  statistic  for  the  Air  Force  Should  Cost  contracts, 
Ua,  and  that  for  the  Army  Should  Cost  contracts,  Ub,  are 
then  calculaced  as  follows: 

Ua  -  (15)  (6)  +  .  130 

*  80 

Ub  -  (15) (6)  +  (6) j6*1*  -  101 
« 

SB  10 

In  this  case,  there  exists  enough  evidence  to  reject  the 
null  hypothesis  and  conclude,  at  the  95  percent  confidence 
level,  that  the  population  associated  with  Ub  is  shifted  to 
the  right  of  that  associated  with  Ua.  It  can  be  concluded, 
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TABLE  3-6 

AIR  FORCE  AND  ARMY  SHOULD  COST  DATA  COMPARED 


Air  Force 

Army 

Contract 

%  Reduction 

Rank 

%  Reduction 

Rank 

'  1 

8.42 

6 

15.28 

16 

2 

13.93 

12 

10.57 

8 

3 

15.16 

15 

18.39 

19 

4 

17.26 

18 

22.09 

20 

5 

15.04 

14 

15.29 

•  17 

6 

14.46 

13 

46.81 

21 

7 

6.43 

3 

8 

7.16 

4 

9 

12.52 

9 

10 

13.37 

10 

11 

13.50 

11 

12 

7.73 

5 

13 

5.92 

2 

14 

9.00 

7 

15 

5.52 

1 
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based  on  the  available  data#  that  the  application  of  Should 
Cost  seems  to  have  yielded  higher  percent  price  reductions 
during  negotiations  for  the  Army  than  for  the  Air  Force. 
This  may  indicate  that  the  more  intensive  studies  conducted 
by  the  Army  give  government  negotiators  a  better  base  from 
which  to  work  than  the  smaller,  less  involved  studies  con¬ 
ducted  by  the  Air  Force.  Of  course,  any  such  statement 
should  be  qualified.  There  are  many  possible  factors  which 
may  have  intervened  because  of  the  method  in  which  data 
were  obtained.  These  will  be  discussed  in  detail  in  Chap¬ 
ter  4.  Also,  one  of  the  observations  from  the  Army  shows  a 
percent  cost  reduction  of  46.81  percent.  There  probably 
were  some  special  circumstances  which  led  to  this  unusually 
larga  reduction.  For  now,  however,  it  probably  is  safe  to 
say  that  there  is  some  evidence  that  Should  Cost  analyses 
conducted  by  the  Army  are  more  effective  at  reducing  con¬ 
tract  costs  than  those  conducted  by  the  Air  Force. 
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CHAPTER  4 

SUMMATION 


This  chapter  contains  a  summary  of  the  research 
efforts  accomplished  by  the  authors  of  this  thesis.  Addi¬ 
tionally,  it  presents  conclusions  based  on  the  tests  per¬ 
formed.  Some  limitations  are  highlighted,  and  recommenda¬ 
tions  are  made.  Finally,  some  areas  for  future  research 
are  suggested  which  may  provide  guidance  to  anyone  inter¬ 
ested  in  studying  some  aspect  of  the  Should  Cost  method. 


Summary 


Should  Cost  is  an  acquisition  technique  first  used 
by  the  Navy  on  an  aircraft  engine  proposal  in  1967.  In 
short,  it  is: 

an  approach  to  cost  analysis  which  challenges  a 
contractor's  cost  proposal,  supporting  data,  and 
rationale  by  integrating  into  a  single,  fully  coor¬ 
dinated  effort  the  auditing,  pricing,  engineering, 
and  management  analysis  of  a  contractor's  manufac¬ 
turing  and  management  operations  [34:1-1]. 

The  overall  purpose  of  Should  Cost  is  to  develop  a  realis¬ 
tic  negotiation  objective  (31:1-1;  34:1-1). 

The  authors  gathered  a  convenience  sample  of  con¬ 
tract  data  from  both  DARCOM  and  AFSC.  The  data  consisted 
of  both  Should  Cost  and  non-Should  Cost  procurement  infor¬ 
mation  on  individual  weapons  systems.  Data  were  used  to 


test  three  hypotheses.  The  first  two  concern  Should  Cost 
and  non-Should  Cost  contracts  for  Air  Force  and  Army  sys¬ 
tems.  The  hypotheses  assert  that  the  percent  cost  reduc¬ 
tions  during  negotiations  are  essentially  the  same  for 
Should  Cost  contracts  as  those  for  non-Should  Cost  con¬ 
tracts.  The  third  hypothesis  involves  a  comparison  of 
Should  Cost  contracts  from  the  Air  Force  and  the  Army.  It 
is  hypothesized  that  there  is  no  difference  in  percent  cost 
reductions  between  the  two  groups. 

Conclusions 


Hypothesis  1 

There  is  no  significant  difference  in  the  percent 
cost  reductions  during  negotiations  between  Air  Force 
contracts  involving  Should  Cost  and  those  where  Should 
Cost  is  not  used. 

Based  on  the  results  of  a  Wilcoxon  Signed  Rank  Test 
and  a  Mann-Whitney  Test,  this  hypothesis  cannot  be  rejected 
at  the  .05  significance  level.  Neither  test  indicated  that 
the  available  samples  provided  enough  evidence  to  conclude 
that  the  percent  cost  reductions  for  Should  Cost  contracts 
were  distributed  any  differently  than  those  for  non-Should 
Cost  contracts.  The  Air  Force  Should  Cost  efforts  are  ap¬ 
parently  not  achieving  significant  reductions  in  acquisi¬ 
tion  costs. 
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Hypothesis  2 

There  is  no  significant  difference  in  the  percent 
cost  reductions  during  negotiations  between  Army  con¬ 
tracts  involving  Should  Cost  and  those  where  Should 
Cost  is  not  used. 

Due  to  the  small  sample  size,  no  quantitative  sta¬ 
tistical  method  was  applied  in  this  case.  However,  a 
qualitative  analysis  of  the  observations  indicated  that 
there  may,  in  fact,  be  some  increase  in  the  percent  cost 
reduction  attributable  to  Should  Cost.  It  appears,  from 
the  information  available,  that  the.  Army's  application  of 
Should  Cost  is  yielding  positive  results.  Caution  should 
be  exercised,  however,  in  accepting  this  conclusion.  The 
size  of  the  samples  and  the  method  in  which  they  were 
gathered  may  render  them  less  than  completely  representa¬ 
tive  of  their  respective  populations. 

Hypothesis  3 

There  is  no  significant  difference  in  the  percent 
cost  reductions  during  negotiations  between  Air  Force 
and  Army  acquisitions  that  employ  the  Should  Cost 
technique . 

The  researchers  found  that,  at  the  95  percent  con¬ 
fidence  level,  the  null  hypothesis  of  no  difference  could 
be  rejected.  There  is  evidence  that  the  Army  application 
of  Should  Cost  yields  higher  percent  cost  reductions  than 
that  of  the  Air  Force.  Ostensibly,  this  is  due  to  the  fact 
that  Army  Should  Cost  studies  are  more  involved  and,  there¬ 
fore,  considered  to  be  more  valid  by  negotiators  for  both 


the  government  and  the  contractors.  However ,  other  factors 
could  be  affecting  the  results.  The  typea  of  weapons,  dif¬ 
ferent  technologies,  different  negotiating  techniques,  and 
different  contractors  employed  by  the  Army  and  Air  Force 
could  all  have  a  bearing.  In  addition,  it  should  be  noted 
that  the  small  amount  of  data  collected  on  the  Army  was 
obtained  from  Should  Cost  coordinators  in  various  organiza¬ 
tions  within  DARCOM.  These  people  probably  have  a  vested 
interest  in  seeing  that  the  Army  continues  to  apply  Should 
Cost.  Of  course,  the  same  could  be  said  of  the  Air  Force 
Should  Cost  data;  but  the  fact  remains  that  the  data  may 
well  be  biased  in  favor  of  Should  Cost.  Accepting  the 
results  of  the  Mann-Whitney  Test  at  face  value,  however,  it 
appears  that  the  Army  is  getting  more  out  of  its  Should 
Cost  studies  than  is  the  Air  Force. 

Limitations 

The  following  is  a  list  cf  factors  which  limit  the 
degree  to  which  the  results  of  this  study  can  be  general¬ 
ized. 

1.  Small  convenience  samples  were  used.  Because 
of  the  general  lack  of  available  historical  data,  the 
samples  used  in  this  study  are  relatively  small.  The 
researchers  were  forced  to  use  that  information  which  could 
be  obtained  through  inquiries  to  various  organizations  in 
the  Air  Force  and  Army.  The  result  is  that  the  samples 
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collected  are  samples  of  convenience.  These  small  conve¬ 
nience  samples  may  or  may  not  be  representative  of  the 
populations  from  which  they  are  drawn. 

2.  Samples  contain  only  Firm  Fixed  Price  and  Fixed 
Price  Incentive  Firm  contracts.  No  data  was  obtained  on 
cost  reimbursement  type  contracts.  This  may  not  be  a 
limiting  factor  since  Should  Cost  has  generally  only  been 
applied  in  cases  where  design  and  production  specifications 
are  well  established.  These  same  contracts  usually  lend 
themselves  to  Fixed  Price  type  agreements.  However,  no 
basis  was  found  for  evaluating  the  effectiveness  of  Should 
Cost  when  cost  reimbursement  contracts  are  used. 

3.  Should  Cost  studies  may  have  an  effect  on  sub- 
sequent-year  procurements.  The  possibility  exists  that  a 
Should  Cost  study  done  for  a  particular  acquisition  could 
affect  negotiations  for  subsequent  buys  of  the  same  weapon 
system.  In  this  study,  however,  all  non -Should  Cost  con¬ 
tracts  were  treated  the  same  whether  or  not  they  were  pre¬ 
ceded  by  a  Should  Cost  contract.  The  size  of  the  available 
samples  did  not  facilitate  an  analysis  of  how  important 
this  factor  is,  but  it  should  be  noted  that  this  may  be  a 
limitation  to  the  study. 

4.  The  use  of  nonpar ametric  tests  does  not  allow 
any  inferences  as  to  the  magnitude  of  the  difference  between 
two  populations.  While  the  conclusion  was  drawn  that  there 


is  some  significant  difference  between  percent  cost  reduc- 
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tions  for  Army  and  Air  Force  Should  Cost  contracts,  the 
size  of  the  difference  is  not  known.  This  fact  may  be 
important.  Army  Should  Cost  studies  are  much  more  in¬ 
volved  than  those  conducted  by  the  Air  Force.  It  follows 
that  the  Army  studies  are  more  expensive.  Unless  some 
determination  can  be  made  as  to  how  much  additional  savings 
result  from  the  Army’s  method,  it  is  not  possible  to  per¬ 
form  any  cost/benefit  analysis  to  help  decide  which  of  the 
two  applications  of  Should  Cost  is  the  more  effective  over¬ 
all.  Again,  however,  due  to  the  small  samples,  the  research¬ 
ers  felt  that  parametric  statistical  tests  could  not  be 
applied  with  confidence. 

5.  Completed  contract  costs  are  not  considered. 

In  gathering  data  for  analysis,  the  researchers  did  not  try 
to  acquire  or  use  information  on  the  completed  cost  of  the 
weapons  systems  procured.  Only  the  contractor  proposed  and 
considered  negotiated  prices  were  considered.  The  reason 
for  this  is  that  the  purpose  of  this  study  was  to  determine 
if  Should  Cost  has  a  significant  impact  on  negotiations. 

No  attempt  was  made  to  analyze  the  effect  of  Should  Cost  on 
the  accuracy  of  cost  estimates.  To  the  extent  that  inaccu¬ 
rate  Should  Cost  estimates  could  cause  contractor  and 
government  negotiators  to  question  the  validity  of  subse¬ 
quent  studies,  this  factor  could  be  significant. 
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Recommendations 

During  this  study,  it  became  apparent  that  the  Army 
is  applying  Should  Cost  in  a  very  organized  manner.  Each 
of  DARCOM's  subcommands  has  assigned  a  Should  Cost  coordi- 
nator  and  staff.  There  are  strict  guidelines  spelling  out 
when  a  Should  Cost  study  is  to  be  done  and  how  it  is  to  be 
performed.  All  Should  Cost  activities  are  under  the  purview, 
of  a  centralized  Should  Cost  office  at  DARCOM  Headquarters; 
but  though  the  Army  seems  to  have  made  positive  strides 
toward  inqplementing  Should  Cost,  there  does  not  appear  to 
be  much  effort  to  maintain  historical  data  on  Should  Cost 
studies.  While  briefing  slides  were  obtained  which  cited 
many  successful  Should  Cost  efforts,  few  of  DARCOM's  sub¬ 
commands  admitted  to  having  such  information  on  file.  If 
there  is  a  centralized  repository  for  contract  cost  data 
for  the  Army,  it  seems  that  the  Should  Cost  coordinators  in 
the  various  EARCQM  organizations  are  not  aware  of  it.  If 
this  is  true,  it  is  very  possible  that  much  valuable  infor¬ 
mation  is  being  lost.  The  researchers  feel  that  a  central 
file  could  be  invaluable  in  determining  the  effectiveness 
of  the  Should  Cost  technique.  If  such  a  tool  exists,  cer¬ 
tainly  its  true  worth  cannot  be  realized  until  all  offices 
concerned  with  Should  Cost  know  it  is  available. 

Unlike  the  Army,  the  Air  Force  does  not  seem  to  be 
very  enthusiastic  about  applying  Should  Cost.  Fewer  Should 


Cost  studies  have  been  completed  by  the  Air  Force,  and 
there  have  been  almost  none  since  1977.  The  researchers 
could  discern  no  formal  structure  to  the  Air  Force 1 s  Should 
Cost  program.  The  most  recent  Air  Force  publication  govern¬ 
ing  Should  Cost  is  AFP  70-7,  dated  1979.  The  decision  on 
when  a  Should  Cost  study  is  to  be  conducted  seems  to  be  made 
rather  arbitrarily,  with  no  clearly  established  criteria  as 
a  guide.  While  more  information  on  past  Should  Cost  studies 
was  available,  it  still  did  not  encompass  all  or  even  a 
majority  of  the  studies  the  Air  Force  has  completed.  The 
researchers  recommend  that  the  Air  Force,  if  it  is  serious 
about  applying  Should  Co3t,  provide  more  structure  and 
guidance  to  its  implementation.  The  rather  haphazard  ef¬ 
forts  to  date  have  probably  not  been  completely  effective, 
and  may  even  have  been  wasteful.  In  addition,  the  recommen¬ 
dation  made  earlier  concerning  maintenance  of  a  central  data 
bank  for  Army  Should  Cost  information  also  holds  for  the 
Air  Force.  If  Should  Cost  is  indeed  a  useful  tool  for 
government  negotiators  to  use,  such  a  file  will  prove  so. 

If  it  is  not  useful,  historical  data  will  bring  that  fact 
to  light. 


Suggested  Areas  for  Future  Research 

The  conclusions  resulting  from  this  research  hardly 
begin  to  answer  all  the.,  questions  concerning  the  real  use¬ 
fulness  of  Should  Cost  as  a  procurement  technique.  There 
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are  numerous  areas  for  which  a  thorough  investigation  is 
warranted.  These  studies  could  be  accomplished  by  the  pro¬ 
curing  activities  or  as  future  thesis  topics. 

The  results  of  this  thesis  indicate  that  the  Army 
may  be  using  Should  Cost  more  effectively  than  the  Air 
Force.  Further  studies  could  analyze  the  approaches  used 
by  the  two  branches  to  determine  what  factors  account  for 
the  differing  results.  Perhaps  the  Air  Force  could  learn 
from  the  Army's  greater  experience  in  the  area  of  Should 
Cost. 

Another  area  for  research  concerns  the  effect  of  a 
Should  Cost  study  on  the  contractor  concerned.  In  view  of 
the  Army  requirement  to  perform  a  Should  Cost  study  on  all 
sole -source  procurements  over  $25  million,  the  contractor 
can  easily  anticipate  such  a  study.  It  would  be  useful  to 
>1.  iw  how  this  affects  the  contractor's  proposal  prepara¬ 
tions,  negotiation  techniques,  and  production  activities; 
or  perhaps  these  Should  Cost  studies  are  simply  accepted 
as  part  of  the  Defer  contracting  game. 

In  the  Air  Force,  Should  Cost  studies  are  accom¬ 
plished  on  a  much  mere  infrequent  basis  than  in  the  Army. 
Because  of  this,  it  might  be  worth  investigating  how  Air 
Force  contractors  react  to  a  Should  Cost  study.  Also,  do 
they  react  any  differently  than  do  contractors  when  the 
Army  conducts  a  Should  Cost  study? 


One  of  the  shortcomings  of  this  thesis  is  the  small 
samples  employed.  Another  study  similar  to  this  one  but 
utilizing  a  greater  number  and  wider  variety  of  weapons 
systems  would  certainly  be  worth  the  time  and  effort  re¬ 
quired.  Certainly,  if  Should  Cost  is  a  waste  of  time  and 
money,  the  DOD  should  be  made  aware  of  this.  Just  as  cer¬ 
tainly,  however,  if  Should  Cost  can  be  of  value  in  the 
acquisition  process,  the  various  branches  of  the  service 
should  concentrate  on  implementing  it. 

The  results  of  this  study  will  no  doubt  be  received 
with  some  degree  of  skepticism,  as  they  well  should  be. 

The  samples  used  are  anything  but  perfect.  Contracts  con¬ 
sidered  are  for  weapons  systems  bought  in  varying  quanti¬ 
ties  in  different  years.  Many  of  the  contracts,  though  for 
the  same  weapons  system,  involved  technologically  different 
items  due  to  design  changes.  Additional  research  is  needed 
to  determine  if  there  is  a  way  to  normalize  data  from  dif¬ 
ferent  production  lots.  If  the  changing  variables  could 
somehow  be  accounted  for  and  factored  out,  a  much  clearer 
picture  of  the  effectiveness  of  Should  Cost  could  be 
obtained . 

In  light  of  the  fact  that  cost  consciousness  seems 
to  be  receiving  a  high  priority  in  the  DOD,  as  evidenced 
by  the  implementation  of  Project  Cost  (1:1) ,  the  Air  Force 
should  be  very  concerned  with  whether  or  not  Should  Cost 
can  be  effective  in  reducing  the  costs  of  acquiring  wea- 
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pons  systems.  This  study,  though  not  conclusive,  casts 
some  doubt  on  the  ability  of  Should  Cost  to  inhibit  cost 
growth  any  more  effectively  than  alternative  cost  analysis 
techniques.  This  is  especially  significant  in  view  of  the 
cost  of  performing  a  Should  Cost  analysis.  It  is,  there¬ 
fore,  the  opinion  of  the  researchers  that  additional  studies 
to  determine  the  effect  of  Should  Cost  with  a  greater  degree 
of  certainty  are  warranted. 
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1.  Reaffirm  acquisition  management  principles. 

.  2.  Increase  use  of  preplanned  product  improvement. 

3.  Implement  multi-year  procurement. 

4.  Increase  program  stability. 

5.  Encourage  capital  investment  to  enhance  productivity. 

6.  Budget  to  most  likely  costs. 

7.  Use  economical  production  rates. 

8.  Assure  appropriate  contract  type. 

9.  Improve  system  support  and  readiness. 

10.  Reduce  administrative  costs  and  time. 

11.  Budget  for  technological  risk. 

12.  Provide  front-end  funding  for  test  hardware. 

13.  Reduce  governmental  legislation  related  to  acquisition. 

14.  Reduce  number  of  DOD  directives. 

15.  Enhance  funding  flexibility. 

16.  Provide  contractor  incentives  to  improve  reliability  and 
support . 

17.  Decrease  DSARC  briefing  and  data  requirements. 

18.  Budget  for  inflation. 

19.  Forecast  business  base  conditions. 

20.  Improve  source  selection  process. 

21.  Develop  and  use  standard  operation  and  support  systems. 

22.  Provide  more  appropriate  Design-to-Cost  goals. 

23.  Implement  acquisition  process  decisions. 

24.  Reduce  DSARC  milestones. 

25.  Submit  MENs  with  service  POM. 

26.  Revise  DSARC  membership. 
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27.  Retain  USDRE  as  Defense  Acquisition  Executive. 

28.  Raise  dolla'r  threshholds  for  DSARC  review. 

29.  Integrate  DSARC  and  PPBS  process. 

30.  Increase  PM  visability  of  support  resources. 

31.  Improve  reliability  and  support. 

32.  Increase  competition. 

G.  Dana  Brabson,  Col.,  USAF.  "Department  of  Defense  Acqui 
sition  Improvement  Program,"  Concepts ,  Autumn,  1981,  p 
55 
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